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ABSTRACT 


This  paper  discusses  the  applicability  of  prediction  markets  in  Defense  Acquisition  projects,  specifically 
in  estimating  their  cost  and  schedule. 

Several  temporal  and  political  factors  can  sometimes  limit  the  effectiveness  of  traditional  methods  of 
project  tracking  and  cost  estimation,  which  may  be  overcome  by  using  prediction  markets.  A  prediction 
market  provides  an  environment  for  traders  to  buy  and  sell  contracts  whose  values  are  tied  to  uncertain 
future  events.  Efficient  prediction  markets  have  been  shown  to  outperform  available  polls  and  other 
forecasting  mechanisms. 

There  are  various  prediction  markets  based  on  different  models  and  algorithms.  Our  focus  is  not  to 
analyze  these  models,  but  to  identify  the  design  principles  of  implementing  a  proven  prediction  market 
model  in  a  defense  acquisition  project.  Some  pilot  studies  have  been  carried  out  that  provide  insight  into 
the  behavior  of  the  market  participants.  We  found  increased  involvement  of  participants  and  greater 
interest  in  the  projects  to  be  major  benefits.  The  areas  that  need  to  be  considered  in  the  design  and 
implementation  of  markets  are  related  to  the  participants  (like,  which  traders  to  include),  the  information 
to  be  collected  or  the  stocks,  the  marketplace  to  be  used  and  the  incentive  structure  to  keep  the 
participants  motivated  to  trade. 

INTRODUCTION 

Historically,  defense  programs  for  weapons  acquisition  have  taken  longer,  cost  more  and  delivered  fewer 
quantities  and  capabilities  than  planned.  (GAO,  2006) 

Some  of  the  reasons  for  such  discrepancies  include: 

•  Lack  of  discipline  in  estimation:  sometimes  all  the  required  factors  or  experienced  people  are  not 
consulted  during  the  estimation  process.  There  is  also  a  lack  of  transparency  and  accountability 
(GAO,  2006) 

•  Unrealistic  expectations:  The  estimates  can  sometimes  be  too  optimistic. (GAO,  2008) 

•  Initial  estimates  tend  to  “anchor”  expectations  (Aranda,  2005):  Initial  figures  of  estimates  set  too 
high  expectations  from  the  start  which  is  difficult  to  meet  later  on.  Also,  it  is  later  on  difficult  to 
incorporate  any  modifications. 

A  recent  study  by  RAND  involved  an  analysis  of  data  in  the  Selected  Acquisition  Reports  (SARs)  for  a 
sample  of  68  completed  programs.  The  results  showed  that  the  average  cost  growth  (including  both  cost 
overruns  and  cost  under  runs)  for  these  programs  was  46  percent  over  the  baseline  estimate  made  at 
Milestone  B  and  16  percent  over  the  baseline  estimate  at  Milestone  C.  While  cost  growth  occurred  earlier 
in  acquisition  projects,  the  development  cost  growth  at  completion  for  programs  initiated  in  1970s,  1980s 
and  1 990s  remained  relatively  steady.  (Arena,  2007) 

It  is  also  worthwhile  to  note  that  of  the  23  programs  assessed  in  the  GAO  report  (GAO,  2006),  around  10 
are  expecting  development  cost  overruns  greater  than  30%  or  have  delayed  the  delivery  of  initial 
operational  capability  by  at  least  1  year.  The  impact  of  such  conditions  is  the  reduction  in  the  value  of  the 
DOD’s  defense  dollars  and  lower  return  on  investment  (ROI)  (GAO,  2006).  See  Appendix  -  Table  1  for 
more  details. 

The  key  recommendation  for  the  executive  action  from  the  report  is  to  ensure  that  appropriate  knowledge 
is  captured  and  used  at  critical  junctures,  specifically  at  the  following  key  points: 

•  Program  start 

•  Design  review  for  transitioning  from  system  integration  to  system  demonstration 

•  Production  commitment 
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An  effective  way  to  capture  knowledge  and  utilize  collective  intelligence  of  all  team  members  is  to  use 
Prediction  Markets  or  Information  Aggregation  markets.  Such  a  market  can  be  used  as  a  supplement  to 
existing  estimation  methodologies  in  generating  more  accurate  cost  and  schedule  estimates. 

PREDICTION  MARKETS:  HISTORY  AND  PURPOSE 

Prediction  Markets  are  emerging  as  a  promising  forecasting  mechanism  for  efficiently  handling  the 
dynamic  aggregation  of  dispersed  information  among  various  agents  (Tziralis  &  Tatsiopoulos,  2007). 
Various  terms  such  as  ‘information  markets’,  ‘decision  markets’,  ‘electronic  markets’  and  ‘virtual 
markets’  can  be  used  to  describe  this.  In  essence,  Prediction  Markets  are  speculative  markets  created  for 
the  purpose  of  predicting  outcome  of  an  uncertain  future  event.  They  provide  an  environment  for  traders 
to  buy  and  sell  contracts  whose  values  are  tied  to  uncertain  future  events. 

In  the  late  80s,  some  academics  at  University  of  Iowa’s  business  school  came  up  with  idea  of  giving 
students  hands-on  experience  in  trading  markets,  such  as  stock  and  commodities.  Instead  of  using  play 
money,  they  created  a  real  market  in  which  anyone  could  bet  modest  sums  on  outcome  of  future  events- 
e.g.  the  next  president  of  the  United  States.  The  Iowa  Electronics  Markets  (IEM)  is  now  a  thriving 
nonprofit  enterprise.  (Tetlock,  2006) 

The  figure  below  shows  a  sample  prediction  market  that  is  commercially  available  (intrade.com,  2009): 


Boeing  787  -  When  will  the  Boeing  787  make  its  maiden  flight? 
Boeing  787  to  make  maiden  flight  before  midnight  ET  on  31  Dec  2009 


Last  Price: 

You  can  buy 
this  at  47.0 

You  can  sell 
this  at  36.5 

|  Explain  | 


|  Trade  Nov. 


Afghan  Politics  -  Who  will  become  President  of  Afghanistan  in  2009  Presidential  Election? 
Hamid  Karzai  to  be  sworn  in  as  President  of  Afghanistan  following  the  2009  election 


Last  Price: 

You  can  buy 
this  at  0.0 

You  can  sell 
this  at  85.0 


80.0 


|  Trade  How 


■  Closing  Price:  80 


(intrade.com) 


Figure  E  A  sample  prediction  market 


The  theoretical  foundations  of  Prediction  Markets  lie  in  the  efficient  market  hypothesis,  which  states  that 
a  sufficient  number  of  marginal  traders  with  rational  expectations,  who  maximize  utility  through 
maximizing  profits,  drive  prices  in  the  market  in  such  a  way  that  there  is  no  opportunity  for  arbitrage 
(Shvarts,  2007). 

It  is  believed  that  if  the  market  is  well  functioning,  contract  prices  reflect  the  collective  wisdom  of  market 
participants.  There  are  three  primary  types  of  prediction  markets  (Wolfers  and  Zitzewitz,  2004): 

•  Winner  Takes  All  Market:  In  this,  the  contract  pays  a  specified  amount  if  and  only  if  the  world 
achieves  some  specific  conditions,  with  the  price  of  the  contract  reflecting  the  market’s 
assessment  of  the  probability  of  that  occurrence. 

•  Index  Market:  In  this,  contract  pays  an  amount  reflected  in  some  condition  of  the  world,  such  as 
the  percentage  of  the  vote  a  candidate  receives  or  the  inches  of  cumulative  snowfall  in  a  city. 

•  Spread  Market:  In  this,  contracts  specify  the  cutoff  that  determines  whether  an  event  occurs  (such 
as  win  margins  in  football).  These  basic  contract  types  can  be  extended  to  generate  additional 
predictions  and  measures  of  market  uncertainty,  such  as  the  probability  that  a  candidate  will 
receive  40%  percent  of  the  vote,  50%,  60%  etc 


Below  table  illustrates  some  examples  for  each: 
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Table  I.  Types  of  Prediction  Markets  (Adapted  from  (Wolfers  and  Zitzewitz,  2004)) 


Contract 

Example 

Details 

Reveals  market 
expectation  of .  .  . 

Winner-take-all 

Event  y:  Al  Gore  wins 
the  popular  vote. 

Contract  costs  %p. 

Pays  $1  if  and  only  if 
event  y  occurs.  Bid 
according  to  value  of  %p. 

Probability  that 

event  y  occurs,  p(y) . 

Index 

Contract  pays  $1  for 
every  percentage 
point  of  the  popular 
vote  won  by  Al  Gore. 

Contract  pays  $y. 

Mean  value  of 
outcome  y:  E[y] . 

Spread 

Contract  pays  even 
money  if  Gore  wins 
more  than  y*%  of 
the  popular  vote. 

Contract  costs  $1.  Pays  $2 
if  y  >  y*.  Pays  $0 
otherwise.  Bid  according 
to  the  value  of  y *. 

Median  value  of  y. 

Prediction  markets  are  most  notably  used  today  for  predicting  election  outcomes,  movie  box  office 
returns,  terrorist  attacks  and  sporting  events  (Zitzewitz,  2004).  Information  markets  have  also  been 
deployed  at  Hewlett-Packard  Corporation  for  making  sales  forecasts,  and  were  found  to  be  more  effective 
than  traditional  methods.  “Not  only  did  the  market  predictions  consistently  beat  the  official  HP  forecasts; 
the  outcomes  predicted  are  consistent  with  the  probabilistic  predictions  of  the  IAM  (Information 
Aggregation  Market).”  (Plott,  2002) 

Google  also  launched  an  internal  prediction  market  in  April  2005.  The  Google  markets  are  similar  to 
Iowa  Electronic  Markets,  and  have  survey  questions  like  “How  many  users  will  Gmail  have?”  Common 
type  of  markets  included  those  forecasting  demand.  (Masse,  2008)  “Google’s  prediction  markets 
are  reasonably  efficient,  but  did  exhibit  four  specific  biases:  an  overpricing  of  favorites,  short  aversion, 
optimism,  and  an  under-pricing  of  extreme  outcomes.  New  employees  and  inexperienced  traders  appear 
to  suffer  more  from  these  biases,  and  as  market  participants  gained  experience  over  the  course  of  our 
sample  period,  the  biases  become  less  pronounced.”  (Masse,  2008) 

Some  of  the  factors  that  are  involved  in  Prediction  Market  are: 

•  Illiquidity:  Prediction  requires  enough  buyers  and  sellers  making  enough  transactions  to  produce 
a  clear  market  price  that  summarizes  the  market  prediction.  In  “thin”  markets  where  buyer, 
sellers  and  their  trades  are  few,  the  prediction  market  can  sometimes  be  ineffective.  Some 
research  suggests  prediction  markets  can  be  effective  with  as  few  as  sixteen  active  traders 
(Christiansen,  2007). 

•  Rationality:  While  prediction  markets  have  been  strikingly  accurate  when  they  meet  the  criteria 
for  effective  markets,  scholars  have  noted  some  deviations  from  “perfect”  rationality.  First,  while 
traders’  preferences  may  bias  their  trades,  such  as  when  partisans  buy  political  futures  contracts 
for  their  favored  candidate  (Forsythe,  1 999),  the  prediction  market  price  of  contracts  will  remain 
accurate  so  long  as  the  enough  marginal  traders  remain  objective  (e.g.,  their  profit  motives 
outweigh  their  preferences).  Prediction  markets  do  not  require  all  traders  be  rational  as  long  as 
marginal  market  exchange  is  motivated  by  rational  traders  (Gruca,  2007),  (Tziralis  &. 
Tatsiopoulos,  2007).  Second,  as  in  horse  racing,  prediction  markets  participants  exhibit  over  price 
long  shots  (so  that  contract  holders  receives  a  smaller  payoff  relative  to  the  true  probability  of  the 
event  occurring)  and  under  price  high  probability  events,  though  (Cowgill  et  al.,  2009)  suggest 
the  precisely  the  opposite  biases  occurred  when  Google  employees  traded  in  internal  company 
prediction  markets  forecasting  Google’s  future.  Moreover,  prediction  markets  may  be  less 
effective  at  accurately  predicting  extremely  likely  or  unlikely  events  (Zitzewitz,  2004) 

•  Manipulation  and  Bubbles:  A  sufficiently  endowed  and  motivated  trader  might  attempt  to 
manipulate  a  prediction  market  by  purchasing  contracts  in  desired  directions.  A  candidate  for 
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elected  office,  for  example,  might  purchase  contracts  on  himself  to  generate  apparent  momentum 
and  publicity.  Speculative  bubbles  are  possible  in  prediction  markets,  a  non-trivial  concern  in 
light  of  recent  macroeconomic  events.  Wolfers  and  Zitzewitz  (2004)  propose  that  a  speculative 
bubble  occurred  in  political  prediction  markets  on  whether  Hillary  Clinton  would  win  the  2004 
Democratic  presidential  primary,  and  bubbles  may  have  occurred  in  experimental  prediction 
markets  reported  in  Plott  and  Sunder  (1988).  To  some  extent,  markets  may  self  correct 
manipulation  and  bubbles  (Strumpf,  2004).  The  best  defense  against  manipulation  is  sufficient 
liquidity  so  that  profit  seeking  traders,  recognizing  that  the  manipulated  market  prices  are 
inaccurate  trade  for  contracts  to  bring  predictions  back  into  line. 

PREDICTION  MARKETS  FOR  DEFENSE  ACQUISITION  PROJECTS 

The  DOD’s  ability  to  meet  its  acquisition  targets  is  becoming  increasingly  critical  as  defense  budgets  are 
reduced  and  expectations  to  deliver  on  time  and  on  budget  remain  high.  Costs  overruns  are  one  aspect  of 
this  issue.  Please  refer  to  table  in  Appendix  -  Table  2  for  a  sample  of  cost  overruns  in  large  federal 
projects. 

A  recent  GAO  report  (Edwards,  2009)  concluded  that  “weapon  programs  are  taking  longer,  costing  more 
and  delivering  fewer  capabilities  than  originally  planned.”  It  also  noted  that  “systematic  problems  both  at 
the  strategic  and  at  the  program  level”  (GAO,  2008)  were  to  blame.  The  GAO  also  noted  that  military 
branches  “overpromise  capabilities  and  underestimate  costs  to  capture  the  funding  needed  to  start  and 
sustain  development  programs.”  (GAO,  2008) 

In  2008,  GAO  reviewed  cost  and  schedule  of  72  weapons  programs  and  found  that  the  average  cost 
overrun  for  systems  development  was  40%  (GAO,  2008)  and  concluded  that  “DOD’s  acquisition 
outcomes  appear  increasingly  suboptimal”  (GAO,  2008).  A  study  by  Deloitte  consulting  also  agrees  that 
defense  cost  overruns  are  getting  worse.  (Irwin,  2009) 

The  table  below  gives  a  brief  comparison  between  existing  cost  estimation  techniques  and  the  prediction 
markets.  The  goal  here  is  to  compare  the  probable  shortcomings  of  the  former,  with  the  possible 
advantages  of  the  latter. 


Table  II.  Value  propositions  of  Prediction  Markets 


Probable  limitations  of  existing  cost  methods 

Possible  value  propositions  of  Prediction 
Markets 

•  Cost  Estimating  Relationships  (CERs) 
based  on  technical  factors  rather  than 
programmatic  “soft”  factors. 

•  Shift  of  focus  from  estimating  by 
individuals  to  groups. 

•  Not  dynamically  undated  as  a  program 
evolves,  making  the  original  estimate 
outdated  as  soon  as  the  climate  changes. 

•  Provide  way  to  leverage  information  from 
diverse  sources. 

•  Manifestation  of  a  few  decision  makers, 
under  tremendous  time  pressure,  working 
with  limited  and  perhaps  biased 
information. 

•  Mitigate  biases  stemming  from  pressure  to 
‘price  to  win’  or  hide  damaging 
information. 

•  Enable  freauent  sampling  of  information. 

•  Incentivize  traders  to  seek  out  quality 
information  that  will  help  them  do  better. 

Prediction  markets  are  a  judgmental  forecasting  method  (Graefe,  2009)  where  the  price  mechanism  of  the 
market  automatically  aggregates  all  the  dispersed  information  among  participants.  Various  aspects  of  the 
prediction  markets  are  also  discussed  in  further  sections. 
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DESIGN  PRINCIPLES  OF  PREDICTION  MARKETS 


As  with  any  markets,  prediction  markets  may  fail  -  and  produce  inaccurate  forecasts  -  if  not  properly 
designed  and  executed.  They  should  follow  specific  mechanisms,  contracts  and  contexts  to  ensure 
participants  are  adequately  incentivized  to  participate. 

Some  other  factors  to  be  incorporated  are  (Suda,  2009): 

•  Should  require  unbiased  tacit  knowledge  of  crowd 

•  Should  be  designed  in  well-  incentivized  environments 

•  Should  be  designed  for  a  non  -  hierarchical  environment 

•  Generally  not  suitable  for  idea  generation  process 

•  Should  have  well  defined  end  dates  or  closing  criteria 

•  Choices  should  be  mutually  exclusive  and  have  a  definitive  outcome 

PILOT  STUDIES  ON  PREDICTION  MARKETS 

In  order  to  study  the  prediction  markets,  we  conducted  some  simulations  on  a  group  of  students  at  MIT  to 
study  the  effectiveness  of  prediction  markets,  and  some  interesting  observations  were  noted.  Details  of  the 
study  are  given  below: 

Pilot  Study  1:  Predicting  ratings  for  Sunday  Night  Football 

A  group  of  17  participants  (N=17)  were  asked  to  participate  in  a  private  marketplace  for  predicting  the 
Nielsen  Rating  (NBC  -  USA)  for  Sunday  Night  Football  (Oct  25).  Please  refer  to  Appendix  -  Figure  1,  2, 
3  for  the  screenshots  of  the  market.  The  following  rules  were  set  in  place: 

•  Time  duration  for  trading  was  5  days,  with  the  market  set  to  close  30  minutes  prior  to  game  start. 

•  Each  participant  allotted  an  initial  amount  of  $5000  for  trading  (play  money). 

•  Winning  Criteria  was  generating  maximum  worth. 

•  Incentive  -  Points  for  class  participation  grade  and  a  popular  book  for  winner. 

•  Minimum  of  four  trades  per  person  (tied  to  the  incentive  of  class  grade),  with  no  maximum  limit 
on  trading. 

•  11  rating  ranges  were  present  as  choices  or  contracts  for  participants  to  trade  on,  with  each  rating 
range  being  assigned  equal  probability  (9.09%  each)  to  start  with  (See  Figure  2). 

•  Participants  could  ‘buy’  or  ‘sell’  shares  of  the  choice  or  contract  they  found  most  likely  to  occur. 
Every  time  a  user  bought  a  share  for  a  particular  idea,  the  price  went  up.  Similarly,  each  time 
users  sold  a  share,  the  price  went  down.  The  User’s  account  was  also  credited  or  debited  based  on 
his/her  choice. 

•  There  was  a  direct  correlation  between  price  and  probability  -  share  price  of  $9.09  indicated 
9.09%  chance  of  that  particular  rating  range. 

•  The  buy  and  sell  transactions  were  managed  by  an  automatic  market  maker. 

Results: 

Around  236  trades  were  conducted  in  total  by  17  participants. 

The  highest  rating  range  predicted  was  of  (9.6-10)  with  a  19%  likelihood  prediction,  followed  by  range 
(9. 1-9.5)  with  a  17.3%  chance  (See  Figure  3). 
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What  will  be  the  Nielsen  Ratings  for  the  NBC  Sunday  Football 
Game  (Oct25)? 


TIP:  CunM  «  probetalty  pr*: 

k*m  «■  cccv  0 1  S10  ■  10%  duncp  pixkt on  w*  o cat 

Select  a  prediction 

Between  9  6-10  0 

$19  08 

(dosed) 

Betwoon  9  1-95 

$17  28 

(dosed) 

Botwooo  10  1*105 

$12  69 

(dosed) 

BoNmkki  8  0-8  5 

$11  57 

(dosed) 

Bfrfwooo  11  1*12  0 

$11  44 

(closed) 

BoNroen  10  6  *  11  0 

$9  90 

(closed) 

Between  8  6-9  0 

$6  58 

(closed) 

Between  1 2  0  -  13  0 

$365 

(closed) 

Below  80 

$2  85 

(closed) 

CVer  16  0 

$2  48 

(closed) 

Between  13  0  -  16  0 

$2  43 

(closed) 

(inklingmarkets.com) 

Figure  2.  The  stocks  in  the  prediction  market 


The  actual  rating  range  turned  out  to  be  (9. 1  -  9.5) 

Actual  Range 

Predicted  Range  (9.1 -9.5) 
(9.6-10.0)  ▲ 


Figure  3.  Results  at  the  close  of  prediction  market 


A  Probability  distribution  function  (Pdf)  for  the  result  is  shown  in  Figure  4,  which  demonstrates  how 
close  the  predicted  result  was  to  the  actual  result. 
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Predicted  Range 


Figure  4.  Pdf  of  the  prediction  market  results 


Although  market  predicted  range  was  close,  there  can  be  two  reasons  why  the  market  forecast  was  not 
absolutely  accurate: 

•  On  the  day  of  the  game,  a  World  Series  game  ran  into  overtime  -  overlapping  with  the  football 
game  and  impacted  the  expected  ratings  for  the  Sunday  night  football.  It  turns  out  that  there  were 
no  baseball  fans  in  the  participating  traders,  and  hence  this  information  was  not  captured. 

•  In  a  study  on  markets  on  NFL  football  games  from  tradesports.com,  there  was  an  observation  of 
mispricing  (Solomon,  2008)  visible  primarily  in  sporting  markets.  This  was  due  to  a  behavioral 
bias  known  as  disposition  effect,  which  is  the  tendency  of  investors  to  sell  stocks  which  have 
risen  in  value  and  hold  on  to  stocks  that  have  fallen  in  value,  with  the  objective  of  earning 
maximum  profit.  Thus  there  might  be  mispricing  in  such  scenarios  and  the  hypothesis  that  the 
prices  in  prediction  markets  reveal  unbiased  estimates  of  true  probabilities  of  the  event  happening 
might  not  be  true. 

Pilot  Study  2:  Predicting  ratings  for  Monday  Night  Football 

Based  on  the  previous  pilot  study  feedback,  the  structure  of  the  next  market  was  modified.  The 
participants  (N=13)  were  asked  to  predict  the  Nielsen  Rating  (Cable  TV  -  US)  for  Monday  Night  Football 
on  ESPN  .Instead  of  providing  range  choices  to  participants  for  bidding,  they  were  asked  to  bid  on  an 
indexed  market.  Rules  were  similar  to  the  last  market,  except  for  the  following  changes: 

•  Instead  of  ranges,  participants  now  saw  only  one  rating.  They  had  to  predict  if  the  rating  as  per 
their  opinion  was  to  be  higher  or  lower.  Based  on  their  responses,  the  price  or  rating  was  adjusted 
automatically. 

•  The  initial  rating  set  was  10. 

•  The  price  and  rating  scale  was  1:1,  i.e.  a  price  of  $10.00  meant  the  current  projected  value  of 
rating  was  10.00  (See  Figure  5). 

Results: 

Around  116  trades  were  conducted  by  13  participants.  The  final  rating  predicted  by  the  market  was  10.00 
(See  Figure  6) 
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What  will  be  the  Nielsen  Rating  (Cable  TV  -  US)  for 
Monday  Night  Football  on  ESPN  (Nov2)? 


Rating  $10.00 

Do  you  iTink  me  value  ml  be 

*  Greater  tian  1 0  00 

•  low  Ihan  1000 


Q  AMI  tm  atan  tfw  pnc*  r»  jrNtw  Tun  12  0  an*  or  Ini  Bun  •  0  (jbi  Im4  Mp 

Figure  5.  The  stocks  in  the  prediction  market  (inklingmarkets.com) 


Nielsen  Ratings 


o  • 


Figure  6.  Graph  showing  change  in  value  of  the  stock  upon  trading 

There  was  a  huge  variation  in  the  ratings  predicted,  with  the  highest  value  being  33.1  and  the  lowest  value 
being  0.6 

The  actual  rating  turned  out  to  be  7.7  which  were  different  from  the  market  value.  The  major  reason  for 
this  was  that  market  place  was  too  volatile.  A  single  participant  could  buy  a  large  number  of  shares,  and 
alter  the  rating  to  a  great  extent,  which  is  not  a  good  design  principle  for  prediction  market. 

Observations: 

Behavioral  observations  of  Participants: 

The  participants  comprised  a  mix  of  people  with  football  knowledge  and  others  with  no  knowledge  at  all. 
The  actual  result  of  the  rating  was  quite  close  to  the  rating  range  that  the  test  market  predicted.  But  the 
main  benefit  was  observed  in  the  behavior  of  the  participants.  An  unbiased,  incentivized  market  piqued 
their  interest  and  motivated  them  to  remain  involved  throughout  the  market  duration.  Some  other 
observations  were: 

•  Increased  participation  due  to  Internal  competition 

•  Attempts  at  market  gaming 
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•  Trying  to  make  use  of  First  mover  advantages 

•  Forming  alliances  to  manipulate  the  market 

•  Fligh  incentive  to  participate  led  participants  to  gain  more  knowledge  on  the  subject. 

•  Use  of  statistical  models  by  some  participants  to  forecast  ratings 

Other  useful  insights  from  pilot  studies  for  implementing  prediction  markets: 

•  Ensure  enough  liquidity:  Research  done  on  prediction  markets  (Christiansen,  2007)  suggests  at 
least  16  participants  for  an  effective  market.  Based  on  the  two  pilot  studies,  the  number  of 
participants  was  lesser  in  the  second  study  potentially  explaining  the  volatility  and  resulting 
significant  error  in  the  prediction. 

•  Avoid  volatility  by  choosing  the  right  scale  for  trading  in  market:  The  market  needs  to  be  less 
volatile,  so  that  it  is  not  easily  influenced  by  a  single  trader. 

•  Ease  of  Use:  The  Prediction  market  should  be  easy  to  use,  as  many  participants  were  initially 
reluctant  to  trade  due  to  perceived  complexity. 

•  Ensure  regular  information  update:  In  both  the  studies,  there  were  not  enough  ‘happenings’ 
regarding  the  game  ratings,  during  the  trading  duration  to  sustain  participants’  interest.  The 
market  is  expected  to  fare  even  better  is  some  new  knowledge  is  available  throughout  the  trading 
duration. 

•  Leads  to  increased  awareness  and  interest:  A  significant  contribution  of  prediction  markets  in  the 
field  of  cost  or  schedule  estimation  is  to  keep  the  participants  involved  and  make  the  process 
exciting.  Traders  with  no  knowledge  of  football  got  involved  by  researching  more  on  the  subject. 

•  Align  incentives  with  the  participants’  interests:  Some  people  just  executed  the  minimum 
required  four  trades  and  stopped.  Although  there  was  high  trading,  some  people  did  not  care  to  do 
all  the  research.  One  reason  for  this  was  due  to  incentives  not  aligning  directly  with  the  objectives 
of  these  participants. 

DESIGNING  PREDICTION  MARKETS  FOR  DEFENSE  ACQUISITION  PROJECTS 

There  are  various  components  of  a  market  that  need  to  be  considered  prior  to  designing  and  implementing 

a  prediction  market  for  a  project.  A  prediction  market  is  composed  of  the  following: 

•  Stocks:  These  are  the  outcomes  or  possibilities  of  the  market  that  are  collectively  exhaustive  and 
mutually  exclusive.  These  act  as  stocks  and  securities  for  the  traders  to  trade  against,  with  a  ‘buy’ 
indicating  a  belief  of  occurrence  of  that  particular  outcome  and  a  ‘sell’  indicating  a  belief  of  non¬ 
occurrence. 

•  Marketplace:  This  refers  to  the  software  environment  which  contains  the  market  or  the  questions 
for  trading.  It  also  encompasses  the  market  manager  which  facilitates  trades  among  the  traders 
and  manages  the  information  distribution  process. 

•  Traders:  These  are  the  participants  or  team  members  in  a  project  that  will  be  trading  information 
in  the  form  of  stocks  in  the  market. 


Market  Place 

Figure  7.  The  composition  of  the  prediction  market 
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There  needs  to  be  a  continuous  flow  of  information  for  the  traders  to  base  their  decisions  on  and  trade 
throughout  the  market  duration. 

Incentive  structure  is  another  important  factor  that  should  be  considered  during  market  design,  as  it  acts 
as  the  motivating  factor  for  the  traders. 

The  success  of  the  project  hinges  on  identifying  an  acquisition  program  willing  to  participate.  The  ideal 
characteristics  of  the  program  are: 

•  uncertain  cost  and  schedule  components  that  can  be  specified  clearly  in  a  contract 

•  a  sufficiently  large  program  community  to  ensure  a  liquid  market 

•  ample  “soft”  relative  to  “hard”  information  and  information  is  broadly  and  unevenly  held  by 
diverse  actors 

•  one-of-a-kind  program  or  a  program  with  limited  relevant  historical  information 

•  susceptible  to  performance  impacts  as  a  result  of  external  events  (i.e.,  political  landscape,  policy 
changes,  personnel  attrition,  technology  maturity,  design  modifications) 

Based  on  initial  pre-screening,  we  identified  ‘Shallow  Water  Combat  Submersible’  (SWCS)  as  a  suitable 
project,  with  a  plan  to  implement  the  prediction  market  for  duration  of  3-6  months. 

Below  is  some  of  the  discussion  on  the  design  components  for  the  prediction  market: 

1)  Stocks:  the  stocks  or  the  outcomes  traded  in  the  prediction  market  form  the  most  important 
component  of  the  market.  The  stocks  traded  will  give  us  an  indication  of  the  beliefs  of  the  participants, 
and  the  number  of  stocks  traded  will  show  us  the  level  of  confidence  in  their  beliefs. 

Hence  the  questions  that  are  asked  in  the  market  are  very  important. 

There  are  mainly  two  categories  of  questions  that  can  be  asked: 

•  Type  I  questions:  Asking  these  questions  can  directly  provide  the  required  information.  Q,  ->A 

•  Type  II  questions:  Asking  these  questions  can  indirectly  provide  the  required  information,  and  a 
series  of  questions  might  be  required  for  getting  the  information  directly.  Q2  Q’  Q”  A 

This  is  similar  to  another  perspective,  as  per  which  the  state  of  the  world  can  be  divided  on  basis  of  two 
dimensions  (Healy  and  Ledyard,  2009).  Some  questions  can  based  on  an  unobservable  factor  whose  value 
can  impact  realization  in  second  dimension,  e.g.  the  underlying  monetary  policy  of  a  central  hank  .While 
some  questions  can  be  based  on  a  directly  observable  factor  e.g.  Interest  rate  each  quarter. 

The  latter  type  of  questions  correspond  to  Type  I  questions,  while  the  former  compare  to  Type  II 
questions. 

Handling  Type  II  questions: 

The  distinction  of  questions  based  on  these  categories  is  important  because  these  allow  us  to  focus  more 
carefully  on  the  ‘stocks’  or  outcomes  of  Type  II  questions,  so  that  the  required  information  is  available. 

•  The  suitable  approach  with  the  Type  II  questions  is  to  ask  multiple  questions  in  the  first  round  of 
prediction  market,  and  subsequently  drill  down  to  specific  questions  in  further  rounds. 

•  If  we  consider  a  single  question  as  a  single  market,  then  one  of  the  approaches  can  be  to  run 
multiple  markets  at  a  given  time,  and  then  to  add  follow-up  questions  based  on  the  responses. 
New  markets  (i.e.  questions)  can  be  run  every  week,  and  old  questions  can  be  updated  weekly 
based  on  new  information.  The  important  factor  with  this  approach  is  to  decide  which  questions 
need  to  be  deployed  first. 

Designing  questions  for  the  market: 

There  is  a  lot  of  information  that  can  prove  useful  in  case  of  a  defense  acquisition  project.  This 
information  can  be  either  related  to  any  factors  impacting  the  cost  or  schedule  of  the  project,  or  can  be 
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related  to  the  current  or  future  decision  processes  within  the  project.  Based  on  the  type  of  information 
required,  we  have  further  categorized  the  market  questions: 

•  Questions  of  the  First  Order:  These  are  the  questions  that  give  us  the  information  about  the 
program  of  interest.  Some  examples  of  questions  for  the  pilot  project  are: 

o  Will  SWCS  be  certified  by  August  1,  2010? 
o  The  cost  of  the  first  unit  will  be  $x. 

o  The  final  cost  of  the  program  will  exceed  the  baseline  estimate  by  x%. 

•  Questions  of  the  Second  Order:  These  are  the  questions  that  give  us  the  information  about  traders 
and  the  trading  process.  For  the  pilot  project,  some  of  the  questions  are: 

o  What  is  your  motivation  for  trading?  (e.g.  to  win/  to  solve  the  problem/  to  validate  my 
knowledge) 

o  What  is  your  role  in  the  Organization? 
o  What  are  useful  sources  of  information  about  the  project? 

•  Questions  of  the  Third  Order:  These  questions  give  us  information  of  the  behavior  outside  of  the 
prediction  markets.  Some  of  the  questions  are: 

o  Flow  have  you  used  the  prediction  market  in  your  job?  If  you  notice  a  change  in  a 
contract  price,  how  have  you  used  that  information? 
o  Did  trading  improve  your  confidence  in  your  opinion? 
o  Did  Prediction  Markets  encourage  you  to  change  your  communication  habits? 

The  first  order  questions  will  be  asked  in  the  prediction  market  for  trading.  The  second  order  and  third 
order  questions  are  more  of  profiling  questions  which  can  help  decide  how  to  improve  the  markets,  or 
how  the  participants  are  gaining  from  the  market  information.  These  questions  can  be  sent  to  the 
participants  as  a  survey  and  can  be  tied  to  incentives  so  as  to  encourage  them  to  participate.  Incentive  can 
be  additional  bonus  money  (like,  Earn  $100  more  (of  play  money)  for  answering  5  questions) 

The  above  factors  helped  to  shortlist  some  of  the  questions  to  be  used  in  the  prediction  market  for  the 
pilot  defense  project  (SWCS).  A  risk  mapping,  showing  the  risk  covered  by  a  specific  question  of  the  first 
order,  along  with  the  modified  version  of  the  question  is  shown  below.  Note  that  the  first  four  questions 
are  of  type  I,  while  the  last  question  is  of  Type  II. 

Table  III.  Risk  Mapping  for  First  Order  Questions 


Question 

Type 

First  Order  Questions 
(information  about  the  program  of 
interest) 

Measured  Risk 

Alternate  way  of  asking  the  question 

Type  1 

Will  SWCS  be  certified  by  August  1,  2010? 

Schedule  Overrun  ] 

When  will  SWCS  be  certified? 

By  how  many  days  will  the  certification  date  be 
exceeded? 

Type  1 

The  cost  of  the  first  unit  will  be  $x. 

Cost  Overrun 

What  do  you  think  is  the  most  significant  cost  driver 
for  the  first  Unit? 

Follow  up  question: 

Based  on  the  below  ranking  of  cost  drivers  for  the 
first  Unit,  what  do  you  think  the  cost  of  first  Unit  will 
be? 

Type  1 

The  final  cost  of  the  program  will  exceed 
the  baseline  estimate  by  x%. 

Cost  Overrun 

Type  1 

What  will  be  the  2011  Fiscal  Year 
appropriation  for  the  SWCS? 

Costand  Schedule 

Type  II 

The  SWCS  program  will  be  adequately 
resourced  to  meet  its  milestones? 

Resource 

Allocation  Errors 

By  how  many  days/months  will  the  milestone  be 
missed? 

Which  resource  do  you  think  is  most  necessary  to 
meet  the  program's  schedule  objective? 

Given  the  available  resources,  will  the  project  be  able 
to  meet  its  planned  schedule? 
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Another  important  aspect  is  to  mix  some  tun  questions  (like  who  will  win  the  Monday  night  game,  etc) 
along  with  serious  questions  to  keep  the  participants  involved. 

•  Marketplace:  There  are  two  important  aspects  of  design  for  the  marketplace  -  the  software 
environment  in  which  the  participants  perform  trades,  and  the  market  manager  which  facilitates  the 
trading  process. 

Considerations  for  the  Software  Environment: 

Following  are  the  design  considerations  for  the  software  environment,  based  on  observed  behaviors  in 
pilot  simulations  and  requirements  for  a  defense  environment: 

•  The  software  environment  will  be  easy  to  access  and  use. 

•  The  software  environment  will  be  commercially  available,  with  prior  success  in  defense  related 
environment. 

•  The  environment  will  satisfy  the  IT  security  considerations  -  like  secure  login,  user  access  rights, 
data  security  etc. 

•  The  software  environment  will  have  dashboard  and  administrative  capabilities  to  analyze  the  data 
and  generate  reports  based  on  the  user  behavior. 

•  It  will  be  possible  to  share  new  information  with  the  traders  and/or  close  or  open  new  markets 
whenever  required. 

Considerations  for  the  Market  Manager: 

The  matching  mechanism  in  prediction  markets  has  most  often  been  a  continuous  double  auction  in  which 
computer  software  matches  buyers  offering  bids  with  sellers  and  their  asking  prices.  Prediction  markets 
have  been  successful  using  real  world  money,  purely  “play  money”  with  no  economic  value  (beyond  the 
satisfaction  of  “winning),  and  “prize  money”  that  can  be  exchanged  for  prizes  and  entries  in  prize 
drawings.  We  will  establish  a  prediction  markets  with  play  and  prize  money  and  will  establish  contracts 
for  the  cost  and  duration  of  particular  features  of  the  acquisition  program.  This  is  more  realistic  than 
focusing  on  the  entire  program  completion  given  the  limited  time  we  have  to  run  the  market.  Fully 
specified,  clear  and  enforceable  contracts  in  prediction  markets  require  the  future  state  of  the  world  (or 
events)  on  which  the  contracts  can  be  easily  adjudicated. 

Market  Scoring  Rule  invented  by  Professor  Robin  Flanson  at  George  Mason  University  will  be  used  in 
the  Prediction  Market.  This  will  ensure  that  fewer  traders  can  be  present  in  the  market,  and  lot  of  concepts 
related  to  stock  markets  that  are  difficult  to  understand  (like  bid-ask  spreads  etc)  can  be  avoided  (Siegel, 
2010).  The  price  of  the  stock  represents  whatever  the  last  trader  was  willing  to  pay  for  shares  and  is  set 
automatically  according  to  the  volume  and  sentiment  in  the  question.  For  example,  if  someone  buys  10 
shares  of  an  answer,  the  price  of  the  share  will  be  increased  automatically  according  to  an  algorithm. 

To  ensure  sufficient  liquidity,  we  will  use  a  prediction  market  with  automated  market  maker,  which 
means  that  the  buying  and  selling  of  stocks  will  happen  between  traders  and  the  market  and  not  between 
the  traders  themselves. 

•  Traders:  While  prediction  markets  require  some  disagreement  among  potential  traders  over  forecast, 
(else  no  trades  would  occur)  excessive  information  heterogeneity  can  be  harmful  to  the  markets 
functioning.  If  some  traders  posses  significant  private  information  (and  hence  become  insiders),  the 
outsiders  may  refuse  to  participate,  ultimately  killing  the  market.  Prediction  markets  are  most  likely  to  be 
successful  if  traders  hold  sufficiently  balanced  information  about  the  event  but  have  differing  beliefs 
about  the  meaning  of  common  information.  It  is  also  critical  that  when  new  information  becomes 
available,  it  will  be  widely  known. 

We  will  recruit  market  participants  from  those  employed  in  the  Acquisition  program  or  those  surrounding 
the  Acquisition  program  (i.e.,  subcontractors,  end  users,  consultants). The  traders  in  the  defense  project 
comprise  of  the  different  team  members  across  different  cross-functional  areas.  There  will  be  many 
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people  in  a  project  who  will  have  limited  information  as  to  what  is  going  on  with  a  program  at  large,  and 
they  might  be  biased  to  their  individual  projects.  Hence,  it  will  be  ensured  that  participants  from  all 
smaller  projects,  as  well  as  people  from  other  departments  are  included  in  the  trading  process.  The 
acquisition  project  will  have  sufficiently  broad  following  to  attract  participants  from  inside  and  outside 
the  government.  These  can  include  administrative  personnel,  sub-contracts,  engineers,  end  users, 
consultants  etc.  The  markets  will  be  implemented  for  duration  of  3  to  6  months,  giving  ample  trading  time 
to  the  participants. 

Sometime,  due  to  fear  of  hierarchy  or  work  environment  pressures,  people  do  not  reveal  uncomfortable 
information  related  to  the  project.  In  the  prediction  market,  anonymity  of  traders  will  be  maintained  so 
that  they  can  disclose  useful  information  without  hesitation. 

•  Incentives:  The  incentive  structure  is  perhaps  the  most  important  component  of  prediction  market  in 
the  DOD  environment.  Incentives  should  be  such  that  they  are  non-biased  and  encourage  participation 
independent  of  any  hierarchical  considerations.  “As  with  any  business  incentive  system,  a  considerable 
challenge  exists  in  choosing  incentives  that  motivate  the  right  behavior.  .  .  With  information  markets, 
incentives  must  serve  a  dual  role:  to  motivate  participation  and  to  motivate  participants  to  provide  truth- 
revealing  opinions.  Incentives  that  satisfy  both  criteria  can  be  difficult  to  define.”  (LaComb,  et  al.,  2007). 
Performance  based  incentives  in  the  project  can  be  provided  in  various  ways.  For  example,  participants 
can  be  rewarded  based  on  the  portfolio  value  at  the  closure  of  the  market.  Awards  can  also  be  given  to  the 
most  active  traders,  or  the  traders  contributing  the  most  valuable  information. 

Non-monetary  awards  can  be  as  effective  as  monetary  awards  when  used  as  incentives.  For  example, 
Google  found  that  in  their  internal  markets,  participants  cared  more  about  non-cash  prizes  like  T-Shirts 
rather  than  cash  prizes  (Coles, Lakhani  and  McAfee,  2007).  Similarly,  by  announcing  user  rankings,  the 
play-money  markets  in  Christiansen’s  field  experiment  (Christiansen,  2007)  were  successful  even  without 
monetary  incentives. 

Other  factors  like  social  competition,  recognition  and  opportunity  to  contribute  towards  the  project  can 
also  act  as  important  motivating  factors. 

For  the  pilot  defense  acquisition  project,  we  feel  the  following  incentives  will  encourage  participation: 

•  Declaring  winners  based  on  the  maximum  portfolio  value  at  market  closure. 

•  Maintaining  a  leadership  board  displaying  the  top  players. 

•  Including  results  of  the  prediction  market  in  regular  status  reports  to  track  any  new  or  less  visible 
information. 

CONCLUSION 

In  his  book  The  Wisdom  of  Crowds  James  Surowiecki  (2004)  argued  that  if  we  take  a  crowd  of  diverse 
people  and  correctly  aggregate  their  judgments,  we  will  be  able  to  get  more  accurate  results  based  on  the 
collective  intelligence  of  the  crowd.  This  wisdom  of  crowds  can  be  seen  in  action  every  day 
(Coles, Lakhani  and  McAfee,  2007)  with  the  collective  intelligence  also  driving  Wall  Street  -  the 
probabilities  generated  from  the  market  displayed  through  the  stock  prices. 

The  prediction  markets  can  be  implemented  for  defense  acquisition  projects  to  aggregate  the  dispersed 
information  across  different  functional  areas.  This  information  can  help  keep  track  of  any  cost  or  schedule 
variances,  and  also  identify  any  potential  risks  that  can  impact  the  project.  We  anticipate  prediction 
markets  can  outperform  existing  Defense  Acquisition  estimation  techniques  (i.e.,  parametric,  analogy, 
activity-based  (Boehm,  et  al.  2005))  in  cases  where  there  is  ample  “soft”  relative  to  “hard”  information 
and  information  is  broadly  and  unevenly  held  by  diverse  actors. 

Markets  need  to  be  compatible  with  the  political  and  regulatory  contexts  in  which  they  operate.  For 
example,  federal  regulations  permit  play  and  prize  money  prediction  markets  but  have  restricted  real 
money  markets.  Perhaps  the  most  notorious  prediction  market  was  the  ill-fated  DARPA  sponsored  policy 
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prediction  markets  (Hanson,  2007)  which  collapsed  from  a  political  firestorm  over  the  (quite  illogical) 
proposition  that  terrorists  might  benefit  from  trading  in  terrorism  futures. 

Based  on  the  various  case  studies  and  literature  available  on  prediction  markets,  our  observations  through 
simulations  of  markets,  and  discussion  on  various  aspects  of  defense  acquisition  projects,  we  consider  the 
following  design  principles: 

•  The  acquisition  project  should  have  sufficiently  broad  following  to  attract  participants  from  inside 
and  outside  the  government.  These  can  include  sub-contracts,  end  users,  consultants  etc. 

•  Although  play  money  should  be  used  for  participation  in  the  markets,  the  traders  can  be 
incentivized  based  on  their  constant  participation  and  ability  to  predict  accurate  results. 

•  Ample  historical  data  should  be  available  on  similar  projects,  so  that  participants  who  are  new  to 
the  project  have  some  basis  of  gaining  knowledge  and  predicting. 

•  The  questions  for  the  market  should  be  very  carefully  designed  so  that  the  required  information 
can  be  revealed.  ‘Fun’  questions  should  be  mixed  with  ‘hard’  questions  so  that  the  participants 
remain  involved. 

•  To  ensure  sufficient  liquidity,  prediction  markets  with  automated  market  makers  should  be  used, 
and  at  least  15-16  participants  should  be  targeted  for  trading  (Christiansen,  2007). 

•  Results  of  the  prediction  markets  should  be  carefully  studied  and  reported  as  part  of  regular  status 
reports. 

•  Anonymity  of  participants  with  respect  to  responses  should  be  maintained  to  encourage 
information-sharing. 

•  The  length  of  the  market  and  closure  of  each  specific  question  need  to  be  given  proper 
consideration  while  designing  the  market.  The  support  of  senior  management  in  Defense  projects 
is  essential  for  starting  the  prediction  market. 

•  Markets  need  to  be  compatible  with  the  political  and  regulatory  contexts  in  which  they  operate. 

As  part  of  the  ongoing  research  project,  the  observations  and  feedback  from  preliminary  and  literature 
research  will  be  used  to  run  the  field  trial  with  the  DoD  (Department  of  Defense)  acquisition  programs  for 
a  fixed  duration  (up  to  6  months).  The  prediction  market  for  the  selected  pilot  acquisition  project  still 
needs  to  be  implemented,  and  other  design  and  implementation  factors  will  be  added  or  the  current  ones 
revised  based  on  new  information. 

If  prediction  markets  improve  in  forecasts  in  the  defense  arena,  as  they  do  in  other  venues,  our  aim  will  be 
to  assist  in  implementing  and  then  studying  additional  markets,  including  those  over  the  longer  term  and  a 
wider  variety  of  programs.  It  is  clear  from  our  research  that  a  significant  contribution  of  prediction 
markets  in  the  field  of  cost  or  schedule  estimation  is  to  keep  the  participants  involved  and  make  the 
process  exciting.  Used  with  right  implementation  guidelines,  these  markets  can  prove  to  be  helpful  in 
capturing  the  collective  wisdom  of  the  project  members  for  the  specific  duration. 
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APPENDIX 


Table  1:  Development  Cost  Overruns  by  Decade 

Figure  1:  Development  Cost  Overruns  by  Decade  (in  Fiscal  Year  2005  Dollars)  and  Key  Reform  Efforts 


Development 
cost  overrun: 

$13  billion 

(30%) 


Development 
cost  overrun: 

$12  billion 

(39%) 


Development 
cost  overrun: 

$15  billion 

(40%) 


Key  Studies  and  Initiatives  Impacting  the  Defense  Acquisition  Process 

•  1 970  Fitzhugh  Commission 

•  1 972  Commission  on  Government 
Procurement 

•  1981  Carlucci  Initiatives 

•  1982  Grace  Commission 

1 986  Packard  Commission 

•  1994  Federal  Acquisition  Streamlining  Act 

•  1996  Clinger-Cohen  Act 

DOD  Acquisition  Policy  Changes 

•  1 971  DOD  5000  policy  established 
a  1 975  Policy  revised 

•  1 977  Policy  revised 

•  1 980  Policy  revised 

•  1 982  Policy  revised 

•  1 985  Policy  revised 

•  1 986  Policy  revised 

•  1 987  Policy  revised 

•  1991  Policy  revised 

•  1 996  Policy  revised 

Source:  DOD  (data);  GAO  (analysts  and  presentation). 


Table  2:  Sampling  of  Federal  Cost  Overruns 


Defense  Development 

Costs  ($2008) 

Original  Estimates 

Final  Estimates 

Global  Hawk  surveillance  plane 

$989m  (2001) 

S3. 7b  (2007) 

Expeditionary  Fighting  Vehicle 

SI. 6b  (2000) 

$3.6b  (2007) 

C-  130J  Hercules 

10.9m  (1996) 

430.3m  (2007) 

Extended  Range  Munitions 

86.9m  (1997) 

500.1m  (2007) 

DDG  1000  destroyer 

2.2b  (1998) 

9.3b  (2006) 

V-22  Osprey  helicopter 

4.0b  (1986) 

12.5b  (2006) 

Armed  Reconnaissance 
Helicopter 

388.3m  (2005) 

7S0.9  (2007) 

Space  Based  Infrared  System 
High 

4.2b  (1996) 

8.5b  (2006) 

Armed  Reconnaissance 
Helicopter 

388.3m  (200S) 

750.9  (2007) 

NPOESS  Satellite  System 

5.0b  (2002) 

7.9b  (2007) 

Other  Defense 

Coastal  Patrol  Ships64 

S220m  (2004) 

$350m  (2007) 

Joint  Stnke  Fighter65 

S232b  (2001) 

S337b  (2008) 

Marine  One  (VH-71) 
helicopters66 

$6.  lb  (200S) 

$11. 2b  (2008) 

Coast  Guard,  NSC  ships,  per 
unit67 

$2S0m  (2002) 

$S36m  (2007) 

(Edwards,  2009 

http://www.downsizinggovemment.org/govemment-cost-overruns) 


(GAO,  2006) 
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FIGURE:  Screenshots  from  the  prediction  market  simulations 


What  will  be  the  Nielsen  Ratings  for  the  NBC  Sunday 

Football  Game  (Oct  25)? 

■  ».  ■  1 3* 

12*  H  B12  ■  toe. 

too 

What  do  you  think? 

SO 

There  is  a  9.1%  chance  "Between  8.0 

00 

-8.5”  will  be  the  answer. 

40 

20 

Oet  IS  Oct  15  Od  17 

Nielsen  ratings  are  audience  measurement  systems 
developed  by  Nielsen  Media  Research.  *1  an  effort  to 
determine  the  autfrence  size  and  composition  of 
television  programming  m  the  United 

(inklingmarkets  .com) 


Buying  Shares 


closi 


•  Buying  shares  means  you  think  this  answer  will  happen. 

•  The  more  shares  you  buy,  the  stronger  you  feel  this  answer  will  happen. 

•  If  the  answer  happens,  you  wi  II  make  money.  If  the  answer  does  not  happen,  you  will  lose  money. 


answer  won't  happen 

purchase  price 

answer  will  happen 

losses 

profits 

•  Your  cost  is  the  price  you  will  pay  for  buying  that  amount  of  shares. 

•  The  estimated  new  price  is  what  impact  you  will  have  on  the  price.  Remember,  by  buying  you  are  saying  the  probability  this 
will  occur  is  higher  than  the  current  price,  so  we  raise  the  price  according  to  how  strongly  you  (and  others)  feel. 


(inklingmarkets  .com) 


Marketplace  Stats 

Marketplace  performance  metrics,  this  week's  leaderboards  and  the  all-time  best  traders. 


Total  number  of  traders 

17 

Total  number  of  trades 
236 

Trades  this  week 

236 

Number  of  markets  running 

0 

Markets  completed 

0 


Most  frequently  traded  market  last  7  days 
no  data  yet 


Most  frequently  traded  market  all  time 

What  will  be  the  Neilson  Ratings  for  the  NBC  Sunday  Football  Game  (Oct  25)? 


Most  trades  this  week 

johnhess  (81) 

Most  trades  all  time 

johnhess  (81) 


Biggest  price  increase  in  last  7  days 

Between  9.6-1 0.0  from  What  will  be  the 
Neilson  Ratings  for  the  NBC  Sunday 
Football  Game  (Oct  25)?  a.  1 09  9% 


Biggest  price  decrease  in  last  7  days 

Between  1 3.0  - 16.0  from  What  will  be  the 
Neilson  Ratings  for  the  NBC  Sunday 
Football  Game  (Oct  25)?  ▼  73.26% 


(inklingmarkets  .com) 
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